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Statistical  complexity  of  neural  spike  trains 

Sarah  Marzen,  Michael  R.  DeWeese  and  Jim  Crutchfield 


Summary.  We  present  closed-form  expressions 
for  the  entropy  rate,  statistical  complexity,  and  pre¬ 
dictive  information  for  the  spike  train  of  a  single 
neuron  in  terms  of  the  first  passage  time  probabil¬ 
ity  distribution.  Our  analysis  applies  to  any  one¬ 
dimensional  neural  model  where  observation  of  a 
spike  causes  the  neuron  to  “reset”  to  some  membrane 
voltage  and  in  which  any  noise  term  is  uncorrelated 
in  time.  We  then  use  these  formulae  to  study  the  lin¬ 
ear  leaky  integrate- and-fire  and  quadratic  integrate- 
and-fire  neurons  driven  by  white  noise  in  the  natural 
spiking  regime.  The  statistical  complexity  [2,  1]  is 
simply  related  to  the  interspike  interval’s  mean  and 
coefficient  of  variation.  The  excess  entropy,  or  the  to¬ 
tal  predictive  information,  is  highest  for  neural  spike 
trains  with  low  interspike  interval  coefficients  of  vari¬ 
ation.  Both  the  statistical  complexity  and  the  excess 
entropy  arise  naturally  in  the  context  of  predictive 
rate-distortion  and  could  be  useful  for  characterizing 
“predictor”  neurons  [3]  that  learn  to  predict  other 
neurons. 

Additional  Detail.  The  statistical  complexity 
C'n ,  predictive  information  Ipred(T ),  and  entropy  rate 
hp  are  basic  metrics  that  can  be  used  to  character¬ 
ize  a  neural  spike  train:  and  Ipred{T )  are  use¬ 

ful  for  characterizing  the  entropy  of  a  neural  spike 
train  and,  thus,  useful  for  calculating  the  mutual  in¬ 
formation  between  the  stimulus  and  a  neural  spike 
train;  Cp  reveal  underlying  dynamical  phase  transi¬ 
tions  and  so  qualitative  changes  in  memory  architec¬ 
ture.  These  quantities  also  offer  fundamental  struc¬ 
tural  interpretations:  the  statistical  complexity  is 
the  amount  of  memory  required  to  optimally  predict 
the  future;  the  excess  entropy  E  =  lirri7’^oc  /pred(T) 
is  the  amount  of  future  information  that  is  pre¬ 
dictable.  Hence,  Cp ,  Ipred(T),  and  hp  can  charac¬ 
terize  the  performance  of  predictor  neurons,  for  ex¬ 
ample  [3]. 

However,  calculating  these  quantities  can  be  chal¬ 
lenging,  except  in  the  structureless  case  of  the  Pois¬ 
son  neuron.  Then,  Cp  and  Ipred{T )  vanish,  and 


the  differential  entropy  rate  in  bits  per  second  is 
hp  =  — Alog2  A,  where  A  is  the  Poisson  process  rate. 
We  present  new  formulae  for  C'M,  Ipred(T),  and  hp 
of  a  neural  spike  train  whose  interspike  intervals  are 
drawn  independently  from  some  distribution  F(t). 
This  includes  neural  spike  trains  of  both  integrate- 
and-hre  neurons  and  spike-response  time  neurons 
driven  by  white  noise.  Denote  the  mean  interspike 
interval  as  (T)  and  define  w(t)  =  ff°°  F(t')dt ',  Z  = 
J0°°  w{t')dt' .  When  the  time  resolution  At  of  the  ob¬ 
server  is  very  small, 
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The  entropy  rate  is  hp  =  lim^oo  hp(t)  and  the  pre¬ 
dictive  information/finite-time  excess  entropy  can  be 
calculated  from  Ipred(T )  =  (hp(t)  —  hp)dt.  The 
key  to  deriving  these  expressions  was  proving  that 
the  causal  states  [2,  1]  of  such  a  neural  spike  train 
were  counts  of  the  time  since  the  last  spike. 

We  present  preliminary  results  for  the  linear  leaky- 
integrate  and  fire  model  and  the  quadratic  integrate- 
and-fire  model  with  77(f)  as  white  noise,  so  that 
the  membrane  voltage  follows  the  equation  A-  = 
f(V)+g(V)rj(t).  When  V  reaches  Vpeak,  then  a  spike 
is  emitted  and  V  is  reset  to  Vreset.  First  passage 
time  probability  distributions  and  their  cumulative 
distribution  functions  were  calculated  empirically  by 
generating  105  trajectories  for  each  set  of  parame¬ 
ters  using  an  Euler  integration  scheme.  The  linear 
leaky  integrate-and-hre  model  can  be  described  by 
Vpeak  =  1,  Vreset  =  0,  g(V)  =  O,  f(V)  —  b  —  V 
with  a  >  0  and  b  e  (—00,00);  we  studied  a  <  5 
and  1  <  b  <  5.  The  quadratic  integrate-and-hre  can 
be  described  by  g(V)  =  a,  and  f(V)  =  b  +  V2  for  a 
range  of  a  >  0  and  b  G  (—00,  00);  we  set  Vpeak  =  100, 
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Fig.  1:  A:  Differential  statistical  complexity  of  a  quadratic  integrate-and-fire  neuron.  C p  peaks 
near  the  dynamical  phase  transition  at  b  =  0.  B:  Excess  entropy  of  integrate-and-fire  neurons.  At 

low  interspike  interval  coefficient  of  variation  (7T),  the  excess  entropy  appears  to  be  a  function  only  of  the 
coefficient  of  variation.  Data  from  linear  leaky  integrate-and-fire  neurons  are  blue  and  data  from  quadratic 
integrate-and-fire  neurons  are  red.  C:  Predictive  rate-distortion  curve  using  finite-time  causal 
states  as  predictive  features.  For  linear  and  quadratic  integrate-and-fire  neurons  with  E  ~  0.3  bits 
and  Cp  ~  12  bits,  assuming  a  time  step  of  At  =  10~4,  most  of  the  excess  entropy  can  be  captured  using  only 
50%  of  its  statistical  complexity.  Linear  leaky  integrate-and-fire  data  is  blue;  quadratic  integrate-and-fire 
data  is  green. 


Vreset  =  —100  and  studied  a  <  5  and  0  <  b  <  5. 
Time  is  in  units  of  the  membrane  time  constant. 

The  differential  statistical  complexity  C M  (ignor¬ 
ing  the  At  component  of  C p  in  Eq.  1)  was  well- 
approximated  by 

C„  »  log2(T>  +  1.4^,  (3) 

where  (T)  is  the  mean  interspike  interval  and  -SL 
is  the  interspike  interval  coefficient  of  variation  for 
both  the  leaky  linear  integrate-and-fire  neuron  and 
the  quadratic  integrate-and-fire  neuron  across  the 
entire  simulated  parameter  space.  The  log2(T)  term 
is  the  statistical  complexity  of  a  periodic  signal  with 
period  (T),  and  the  additional  term  1.4 accounts 
for  the  fact  that  the  signal  is  almost  periodic.  For 
both  types  of  neurons,  Cp  is  maximized  at  the  dy¬ 
namical  phase  transition  boundary;  this  is  shown  for 
the  quadratic  integrate-and-fire  in  Figure  1A.  Figure 
IB  shows  that  the  excess  entropy  E  increases  as  the 
interspike  interval  coefficient  of  variation  decreases, 
which  is  what  one  would  expect  for  an  almost  peri¬ 
odic  signal.  The  differential  entropy  rates  h p  of  the 
integrate-and-fire  neurons  decreased  as  the  intrinsic 
noise  parameter  a  approached  0  and  the  spike  train 
became  a  noiseless  periodic  signal. 

One  does  not  need  to  capture  all  of  the  statis¬ 
tical  complexity  in  order  to  capture  most  of  the 
predictive  information;  the  tradeoff  between  pre¬ 


dictive  ability  and  memory  storage  can  be  quanti¬ 
fied  by  the  predictive  information  bottleneck  frame¬ 
work,  for  instance.  Finite-time  T  causal  states  are 
(suboptimal)  lossy  predictive  features  with  which 
we  can  capture  Ipred{T )  of  the  full  excess  en¬ 
tropy  E  using  only  Cp{T)  =  -  fQT  ^  log  ^0^dt  - 

w{t)dt  , _ w(t)dt  fg 
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k)g: 
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w(t)dt 


log  A  of  the  full  sta- 


(T)  (T)  '  (T)  At 

tistical  complexity  Cp.  The  quadratic  integrate- and- 
hre  neuron  has  a  similar  predictive  rate-distortion 
curve  to  the  leaky  linear  integrate-and-fire  neuron 
at  low  (~  0.1)  interspike  interval  coefficients  of  vari¬ 
ation.  Figure  1C  shows  that,  for  these  neurons,  at 
least  90%  of  the  total  predictability  can  be  captured 
with  a  memory  of  only  ~  50%  of  the  total  statistical 
complexity. 
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